This invention relates generally to network traffic monitoring, and more particularly to providing detection of flow-level network traffic anomalies via abstraction levels.
Safeguarding the availability and reliability of resources in computer networks poses a major challenge for network administrators. Conditions detrimental to a network's performance need to be detected in a timely and accurate manner. Such undesirable conditions are usually termed network anomalies and include attacks and abuse of resources, significant changes of user behavior, and failures of mission-critical servers and devices. Many of these events cannot be described by means of explicit signatures or differ slightly from known anomalies patterns. Signature-based intrusion detection systems are thus likely to fail to detect them. Behavior-based anomaly detection techniques are a complementary approach to address these shortcomings. Their inherent assumption relies on the fact that anomalies are rarely observed in traffic and that if an abnormal event is present, certain characteristics of the network behavior change. An anomaly-based detection system establishes baseline profiles of the normal behavior of a network and flags perturbations thereof as abnormal.
In general, every traffic event leaves traces in distributions of flow level traffic features, such as packet header fields (e.g., IP addresses and service port numbers, TCP flags, etc.) and flow properties (e.g., the number of transmitted packets and octets, flow duration, etc.). Each feature distribution includes of a set of associated components, (i.e., the actual values the feature can take). For example, port numbers 80/http and 22/ssh are components of the “service port” feature. Each component in a feature distribution is subject to variation and may exhibit multiple normal behavior modes (e.g., depending on time of day, application states, user behavior). Many existing techniques apply a pre-processing step to the distributions (e.g., take its sample entropy) to obtain an estimate of its properties. However, precious information may be lost at this early stage before being presented to detection algorithms. Each component in a feature distribution is subject to variation and may exhibit multiple normal behavior modes (e.g., depending on time of day). Early summarization of distributions is therefore likely to miss such individual behavior patterns.